User adaptive display device and method thereof

ABSTRACT

Provided are a user adaptive display device and method thereof. The user adaptive display device includes a plurality of display modules that are jointly or individually controllable and adjacent to each other. The user adaptive display device includes a communication unit, a control unit, and a display unit. The communication unit receives data on user information. The control unit analyzes the user information and changes an image configuration according to the user information. The display unit displays an image with the image configuration changed by the control unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority under 35 U.S.C. 119 and 35U.S.C. 365 to Korean Patent Application No. 10-2009-0114546 (filed on 25Nov. 2009), which is hereby incorporated by reference in its entirety.

BACKGROUND

The present disclosure relates to a user adaptive display device andmethod thereof, and more particularly, to a user adaptive display deviceand method thereof, in which the image quality is adjusted according tothe location of a user.

When a user feels inconvenience in watching TV or viewing a PC monitordue to an excessively large or small size of images or texts on itsscreen, the user directly has to adjust the size of images or the fontof texts.

SUMMARY

Embodiments provide a user adaptive display device and method thereof,in which the display image quality can be automatically adjustedaccording to corresponding user information that include the location,voice, and gesture of a user.

In one embodiment, a user adaptive display device including a pluralityof display modules that are jointly or individually controllable andadjacent to each other includes: a communication unit receiving data onuser information; a control unit analyzing the user information andchanging an image configuration according to the user information; and adisplay unit displaying an image with the image configuration changed bythe control unit.

The user information may include information on a distance from a user,and the control unit may changes the image configuration according tothe distance from the user.

The control unit may change the size of the'image or the size of textsincluded in the image, or may perform conversion between athree-dimensional image and a two-dimensional image, according to thedistance from the user.

The details of one or more embodiments are set forth in the accompanyingdrawings and the description below. Other features will be apparent fromthe description and drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view illustrating the configuration of a useradaptive display device according to an embodiment.

FIG. 2 is a view illustrating a display unit of a user adaptive displaydevice according to an embodiment.

FIG. 3 is a view illustrating an exemplary display unit according to anembodiment.

FIGS. 4 and 5 are views illustrating an example of changing an imageconfiguration according to the age of a user.

FIG. 6 is a view illustrating an example of changing an imageconfiguration according to the location of a user among userinformation.

FIGS. 7 and 8 are views illustrating an example of changing an imageconfiguration according to the distance from a user among userinformation.

FIGS. 9 and 10 are views illustrating another example of changing animage configuration according to the distance from a user among userinformation.

FIG. 11 is a schematic view illustrating the configuration of a useradaptive display unit according to another embodiment.

FIG. 12 is a flowchart illustrating a method of displaying a displaydevice according to an embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Reference will now be made in detail to the embodiments of the presentdisclosure, examples of which are illustrated in the accompanyingdrawings.

Hereinafter, exemplary embodiments will be described in detail withreference to the accompanying drawings.

FIG. 1 is a schematic view illustrating the configuration of a useradaptive display device according to an embodiment.

Referring to FIG. 1, a display device 100 may adjust the configurationof a displayed image using the distance from a user and the age of theuser. That is, the image configuration may be adjusted by changing thesize of a displayed text, changing the size of a display screen, andperforming conversion between two-dimensional and three-dimensionalimages.

The display device 100 may include a display unit 110 on which an imageis displayed, a user input unit 120 receiving commands for operating thedisplay device 100 or configuring the image, a control unit 130, and acommunication unit 140 receiving user information.

Since the display unit 110 can be installed on the surface of a wall inan installation space of the display device 100, the display unit 110may occupy all or a portion of the surface of the wall.

For example, the display device 100 may be installed on one or more sidewalls of a living room or family room, and the display unit 110 may beinstalled on all or a portion of the side walls of the living room orfamily room.

Accordingly, the display device 100 may also be called a display wall ora video wall, but will be described herein as the display device 100.

The display unit 110 may be one large-scale screen that is constitutedby a plurality of sub display modules having a tile shape.

For example, as shown in FIG. 2, a plurality of sub display modules 111to 125 may be connected to form one large-scale display unit 110 andoutput a single image. That is, the plurality of sub display modules 111to 125 constituting the display unit 110 may be jointly or individuallycontrolled.

In one embodiment, the plurality of sub display modules 111 to 125 ofthe display unit 110 may operate as individual display modules,respectively. Accordingly, the plurality of sub display modules 111 to125 may output individual images, respectively.

In one embodiment, the display unit 110 of the display device 110 may beconfigured with a projector for projecting images on a wall surface. Inthis case, light including image information from the projector may beprojected on the wall surface.

Thus, the display device 100 may occupy all or a portion of the wallsurface, and may provide single or multiple images to a user.

The display unit 110 may display a certain image or video, for example,certain contents received through the communication unit 140. Suchcontents may include still images or videos that include two-dimensionalor three-dimensional images.

The communication unit 140 of the display device 100 may receivecontents from an external server or other display devices through wiredor wireless networks, and may display the received contents on thedisplay unit 110. Here, the contents may include various applications,as well as broadcasting signals.

The display device 100 may store contents in a memory unit 150. Thecontrol unit 130 may read the contents stored in the memory unit 150,and may display the read contents on the display unit 110.

Also, various contents including audiovisual materials to be displayedand various user information may be received through the communicationunit 140. Here, the user information may refer to the face, age, gestureand voice of a user, the distance from the user, and the location of theuser, which can be construed as information on the user near the displayunit 110.

In order to obtain the user information, the display unit 110 may beprovided with a camera that can photograph a user, and a microphone thatcan receive a voice of the user. Units for obtaining the userinformation may be called recognition devices.

FIG. 3 illustrates the display unit 110 installed on a wall and a userviewing the display unit 110. A recognition device 300 may be providedin the display unit 110 to obtain user recognition information from theuser.

The recognition device 300, which is a unit for obtaining the userinformation, may include a camera for recognizing the face or gesture ofa user, and a microphone for receiving the voice of the user.

The user information such as image or sound that is obtained by therecognition device 300 may be delivered to the communication unit 140.To this end the recognition device 300 may be wired or wirelesslyconnected to the communication unit 140. In another embodiment, the userinformation that is obtained by the recognition device 300 may bedelivered to an external server, and the corresponding user informationfrom the external server may be redelivered to the communication unit140.

By using the information obtained by the recognition device 300, thedisplay device 100 may configure images (including still images orvideos and menu screen or texts) that are displayed. The configurationof the images may include various settings such as changing the size ofthe images, changing the size of the texts, or determining a sub displaymodule on which the images are to be displayed, according to the userinformation.

The user information may include the present or absence of a user, theface, location, gesture and voice of the user, and the distance from theuser. The control unit 130 may recognize and analyze the userinformation, and may configure the images according to the analyzed userinformation. To this end, the control unit 130 may include or usevarious modules or applications for data-processing such as therecognition of the user information and the analysis of the userinformation.

The recognition and analysis of the user information performed by thecontrol unit 130 are briefly described in the flowchart of FIG. 12.

Hereinafter, the recognition and analysis of the user information bycontrol unit 130, and the configuration of the images using therecognition and analysis of the user information will be described indetail.

In operation S100, the control unit 130 may receive data such as userimages or user voices through the communication unit 140. In operationS200, the recognition of the user information may be performed using thereceived data.

That is, the control unit 130 may receive data including the user image(still images or videos) and the user voices that are received from thecommunication unit 140, and may perform the recognition of the face,voice and gesture of a user using the received data.

The user's face recognition may include extracting a face region of auser from image data obtained by a camera 310 of the recognition device300, and utilizing the data of the extracted face region as userinformation.

The user's voice recognition may include removing ambient noises fromvoice data obtained by a microphone of the recognition device 300,extracting voice data of about 87 Hz to about 1.2 KHz corresponding tothe voice frequency band of a user, and utilizing the extracted voicedata as user information.

The user's gesture recognition may include extracting motions orgestures of a user from video data obtained by the camera 310 of therecognition device 300, and utilizing the extracted user's gestures asuser information.

The control unit 130 may utilize a distance between a user and thedisplay unit 110 among data delivered from the recognition device 300 asuser information. That is, the display device may further include a unitfor estimating the location of a user through the camera 310.

In this case, the control unit 130 may require a reference distance forcomparison or reference in order to calculate a distance from a user,recognized by the camera 310. For example, a reference image obtained byphotographing a user spaced by about 1 meter may be stored in advance,and then the appearance of a user may be recognized from an imagephotographed by the camera 300. In this case, the control unit 130 maycompare the sizes of the user with each other to calculate the distancefrom the user using a ratio of the sizes of the user. The distance fromthe user may be calculated by comparing distances between about three orfour feature points with each other, instead of the entire sizes of theuser. When using the above method, it is possible to calculate thedistance from the user (particularly, the face of the user) even when aportion of the body of the user is covered.

To this end, the control unit 130 may calculate the distance from auser, using the face region or size of the user.

On the other hand, instead of calculating the distance from a user usingthe image of the user photographed by the camera, an infraredlight-emitting/receiving device may be further included as a recognitiondevice. In this case, infrared rays emitting from the infraredlight-emitting/receiving device may be reflected by an object at thefront of the display unit 110, and may return to the infraredlight-emitting/receiving device. For example, the distance from theobject may be measured according to a method that is called a Time ofFlight (TOF) distance measurement method, using the reflection light.The above measurement may be performed on the front side of the displayunit 110 to verify the state of the front side of the display unit 110.For example, when a wall is placed at the front side of the display unit110, the time from the emission of the infrared rays to the reflectionof the infrared rays may be uniform. Accordingly, it can be estimatedthat a user is not located at the front side of the display unit 110. Onthe other hand, when the time from the emission of the infrared rays tothe reflection of the infrared rays may be nonuniform, it can beestimated that a user is located at the front side of the display unit110. In this case, the monitoring of the presence or absence of a userby the infrared light-emitting/receiving device may be performed atcertain intervals of time.

Also, when the camera 310 is an infrared camera, a thermograph methodusing the infrared camera may be used to determine whether an object isa user. In this case, the temperature distribution may be measured usinginfrared energy emitting from a user, and then the measurement resultmay be compared with the distance measurement result measured by theinfrared light-emitting/receiving device described above. Thus, it isdetermined whether the object is a user, by determining the size, shapeand temperature of the object.

In operation S300, the control unit 130 may perform an analysis of therecognized user information. In operation S400, the configuration of theimage may be performed according to the analyzed user information.

The control unit 130 may performed operations such as estimating the ageof a user using the face of the recognized user, indentifying a userusing the voice of the user, performing user commands using the gesturesof a user, estimating a current posture of a user, and analyzing thedistance from a user and the location of the user. However, the term,the user gesture, is not necessarily limited to the motion of the user,and it can also be construed as including the postures (e.g., lying,sitting, and standing postures) of the user who is watching TV.

For example, when the current posture of a user is estimated from aninputted image of the user, information on the user's posture accordingto the contour and eye's location of the user's face may be stored inmemory unit 150. In this case, the control unit 130 may rotate an imagedisplayed on the display unit 110 in the clockwise or counterclockwisedirection by about 90 degrees to allow a user to more comfortably lookat the image even when the user is lying.

The control unit 130 may estimate the age of a user from the recognizedface of the user. In this case, since statistic data on the user's facesaccording to the age of the users or settings for each user may bestored in the memory unit 150 in advance, the control unit 130 mayestimate the age of the user from the recognized face of the user. Theestimation of the user's age may be achieved by the voice of the userthat is obtained by the microphone of the recognition device. That is,the control unit 130 may estimate the age of a user using voice data ofthe user that is received. To this end, the memory unit 150 may includea database such as a statistic voice frequency table according to theages of users.

The control unit 130 may identify user commands according to the usergestures that are recognized. That is, the control unit 130 may analyzewhether a user gesture matches an already-stored gesture, and if so, mayoperate the display unit according to a command preset as a usercommand. For example, when a user makes a gesture of making a circle byraising his hands over his head, and this gesture of the user matches analready-stored gesture, a display unit power-off signal may be generatedas data corresponding thereto.

Thus, data signals corresponding to various user gestures may be storedin the memory unit 150 in advance. Data corresponding to the usergestures may be directly designated by a user. For example, when a lyingposture of a user lasts for about 1 hour, this may be set as the displayunit power-off signal. In this case, when the control unit 130determines that the user lies, a timer may be set, and then the controlunit 130 may power off the display unit 110 when about one hour passes.

The control unit 130 may performed an analysis of the user situation,and may analyze a method of determining the distance from a user as userinformation when there are a plurality of users at the front of thedisplay unit 110 and a method of determining the output of images andsounds according to information on noise around a user. For example,when there are a plurality of users in a user image, the control unit130 may calculate the size of the displayed image using the distancesfrom the respective users, and may allow the image to be displayed in anaverage size using the calculated result. The control unit 130 mayautomatically adjust the volume according to the magnitude of the noisearound the user.

Also, the control unit 130 may manage a user profile or a userpreference, using the user information. For this, the control unit 130may continuously manage users using the memory unit 150, and the memoryunit 150 may include a database in which users are classified accordingto their faces or voices. That is, a user database may be built and maybe continuously managed or updated as the user information including thefacial contour, expression, posture, and motion for each user.

The control unit 130 may manage whether there is a modification historyin the size of images or texts according to the facial expression,posture, and the ambient environment of a user. For example, whenambient noise is greater than about several tens of decibels (dB), andthere is a history in which a user has set the volume of the displayunit 110 to level 20, the history may be stored in the user database ofthe memory unit 150 and may be managed. For example, it may be storedand managed that a user A has watched TV while sitting at a distance ofabout 1 m from the display unit 110. That is, the control unit 130 mayalso establish a database including the distances from recognized usersand the postures of the recognized users. Such a database may enable themanagement of TV-watching pattern, i.e., preference of users. Forexample, when the user A prefers watching television sitting on a sofaspaced from the display unit 110 by about 1 m, the size of image may bedisplayed in an appropriate size when the user A approaches the displayunit.

The TV-watching pattern of a user may be managed by the control unit 130and the memory unit 150. According to the TV-watching pattern, the sizeof an image or the size of texts in the image may be automaticallyadjusted.

The control unit 130 may control images or voices outputted through thedisplay unit 110, using the user information that is analyzed. Inoperation S500, as described above, the configuration may be performedaccording to the user information, and the image may be displayedaccording to the configuration.

FIGS. 4 and 5 are views illustrating an example of changing an imageconfiguration according to the age of a user

When the age of a user is estimated among the user informationrecognized by the control unit 130, the control unit 130 may perform animage configuration and an image display according to the age of a user.

For example, as described in FIG. 4, when a user that is identified fromthe face information of the user is of an advanced age, the control unit130 may operate all of the plurality of sub display modules constitutingthe display unit 110 to display an image on the whole region of thedisplay. However, when there is another image configuration that ispreset by the recognized user, the image may be displayed according tothe preset image configuration.

As described in FIG. 5, when a user that is identified from the faceinformation of the user is of a young age, the control unit 130 mayoperate a portion of the plurality of sub display modules constitutingthe display unit 110 to display the image in a smaller size. When aportion of the plurality of sub display modules of the display unit 110operates, only sub display modules corresponding to the location of theuser may operate. Thus, the location of user may also be used as theuser information. With reference to FIG. 6, the sub display modules thatoperate may be selected according to the location of a user at the frontof the display unit 110. Even in this case, the image may be displayedaccording to an image configuration if there is a different imageconfiguration from the image configuration that is preset by therecognized user.

FIG. 6 is a view illustrating an example of changing an imageconfiguration according to the location of a user among userinformation.

As described above, the control unit 130 may extract the user locationinformation from the user image obtained by the recognition device 300that is provided outside or inside the display device, and may allow theselective operation of the sub display module of the display unit 110 tobe performed according to the location of a user.

As described in FIG. 6B, when the control unit 130 determines that auser is located at the right side of the display unit 110, only the subdisplay located at the front of the user modules may operate among thesub display modules of the display unit 110. However, when the locationof a user is frequently changed, the sub display module may also befrequently changed to the degree of making viewing-video to bedifficult. Accordingly, when the location of the user is not changed fora certain time, the operation of the sub display modules may also befixed.

When there is an image configuration that is preset by a recognizeduser, the control unit 130 may allow an image to be displayed accordingto the preset image configuration.

FIGS. 7 and 8 are views illustrating an example of changing an imageconfiguration according to the distance from a user among userinformation.

The control unit 130 may change the size of an image using a distancefrom the display unit 110 to a user. As described in FIG. 7, when thecontrol unit 130 determines that the distance from the user isrelatively far from the user, for example, about 3 m, the control unit130 may operate all of the sub display module to display an image on thewhole of the display unit 110 for more comfortable viewing.

As described in FIG. 8, when the control unit 130 determines that thedistance from the user is within about 3 m, the control unit 130 maycontrol a smaller number of sub display modules to display the image ina smaller size.

In the adjustment of the image size according to the distance from auser, sub display modules to be selected may be determined according tothe location of the user, using the user location information.

The adjustment of the image size has been described as part of the imageconfiguration using the user information. In addition to the adjustmentof the image size, the image configuration may include increasing orreducing the sharpness of objects in the displayed image. When a user isrelatively close to the front side of the display unit, the sharpnessmay be reduced to mitigate the eyestrain. In addition to the sharpness,the brightness of the image may be adjusted, and the volume may beadjusted according to ambient noise. Also, the image may be rotated in aclockwise or counterclockwise direction by 90 degrees according to theposture of the user.

FIGS. 9 and 10 are views illustrating another example of changing animage configuration according to the distance from a user among userinformation.

During the image configuration, the control unit 130 may change the sizeof texts in an image, using a distance from the display unit 110 to auser. For example, as described in FIG. 9, when it is determined thatthe distance from a user is greater than about 3 m, in order for theuser to easily read texts, the control unit 130 may operate all of thesub display modules to display the texts or an image including the textson the whole of the display unit 110.

As a similar example shown in FIG. 10, when it is determined that thedistance from a user falls within about 3 m, the control unit 130 mayselect a smaller number of sub display modules to display texts in asmaller size.

In the adjustment of the text size according to the distance from auser, sub display modules to be selected may be determined according tothe location of the user, using the user location information.

FIG. 11 is a schematic view illustrating the configuration of a useradaptive display unit according to another embodiment.

A display device 100 according to another embodiment may output athree-dimensional image. That is, the display device 100 may provide athree-dimensional image or menu screen. When a contents providerprovides three-dimensional contents, the display unit 100 may output thethree-dimensional contents through the communication unit 140.

An additional component may be required to output three-dimensionalimages. FIG. 11 illustrates the configuration of a display device thatcan process three-dimensional images, according to another embodiment.The functions of all components of FIG. 11 except a 3D formatter may besimilar to those of FIG. 1.

As shown in FIG. 11, the 3D formatter 160 may be disposed at an outputterminal of an image signal processing unit (not shown). That is, the 3Dformatter 160 may be disposed at an input terminal of the display unit110. The 3D formatter 160 may convert an image processed by the controlunit 130 into a three-dimensional image, and may deliver thethree-dimensional image to the display unit 110. According toembodiments, a separate 3D formatter may be provided to convert an OSDoutput of an OSD generation unit into a three-dimensional output.

The configuration shown in FIG. 11 is merely an example according to oneembodiment, and therefore various known technologies may be employed forthe 3D image processing.

The adjustment of the image configuration according to the userinformation as described above may also be performed on such athree-dimensional image. That is, the size of the three-dimensionalimage, the size of texts included in the three-dimensional image, andthe depth of the three-dimensional image may be adjusted according tothe user information.

For example, the control unit 130 may control the depth of thethree-dimensional image to vary according to the distance from a user.In this case, the control unit may control the degree of projectionfeeling of a portion of the three-dimensional image, or may control animage region where the projection feeling is emphasized, to varyaccording to the location of the user.

When a user adaptive display device according to an embodiment of thepresent invention is used as a home display environment, a living roommay be a common place shared by all family members, in which a user maybe frequently changed. However, since the user profile and the userpreference are continuously managed, various applications fit for userscan be executed.

According to the embodiments of the present invention, the image qualitycan be automatically adjusted according to a user or a current state ofthe user without directly setting the size of an image and so forth bythe user.

Any reference in this specification to “one embodiment,” “anembodiment,” “example embodiment,” etc., means that a particularfeature, structure, or characteristic described in connection with theembodiment is included in at least one embodiment of the invention. Theappearances of such phrases in various places in the specification arenot necessarily all referring to the same embodiment. Further, when aparticular feature, structure, or characteristic is described inconnection with any embodiment, it is submitted that it is within thepurview of one skilled in the art to effect such feature, structure, orcharacteristic in connection with other ones of the embodiments.

Although embodiments have been described with reference to a number ofillustrative embodiments thereof, it should be understood that numerousother modifications and embodiments can be devised by those skilled inthe art that will fall within the spirit and scope of the principles ofthis disclosure. More particularly, various variations and modificationsare possible in the component parts and/or arrangements of the subjectcombination arrangement within the scope of the disclosure, the drawingsand the appended claims. In addition to variations and modifications inthe component parts and/or arrangements, alternative uses will also beapparent to those skilled in the art.

What is claimed is:
 1. A user adaptive display device comprising: adisplay unit including a plurality of sub display modules connected toeach other; a recognition unit installed on a portion of the displayunit and configured to obtain data on a user located at a front of thedisplay device; a control unit configured to: analyze the data on theuser to generate a user information, the user information including atleast one of an age of the user, a location of the user, a distance fromthe user and a gesture or voice of the user, adjust a number of subdisplay modules which are operated among the plurality of sub displaymodules to change a size of a display screen based on the userinformation, the sub display modules which are operated corresponding tothe location of the user, and operate the adjusted sub display modulesto display an image on the adjusted sub display modules.
 2. The useradaptive display device according to claim 1, wherein the control unitchanges a size of a text comprised in the image, or performs aconversion between a three-dimensional image and a two-dimensionalimage, according to the distance from the user.
 3. The user adaptivedisplay device according to claim 1, wherein the control unit performsan operation corresponding to the gesture or voice of the user.
 4. Theuser adaptive display device according to claim 3, wherein the controlunit rotates the display unit toward the user in response to the gestureor voice of the user.
 5. The user adaptive display device according toclaim 1, further comprising: a user input unit configured to receive aninput command from the user; and a memory unit configured to store theuser information and the image.
 6. The user adaptive display deviceaccording to claim 1, further comprising a three-dimensional formatterconfigured to convert a two-dimensional image processed by the controlunit into a three-dimensional image, wherein the control unit changes adepth of the three-dimensional image according to the user information.7. The user adaptive display device according to claim 1, furthercomprising: a communication unit configured to connect the recognitiondevice and configured to receive the data on the user from therecognition device or an external server.
 8. The user adaptive displaydevice according to claim 7, wherein the data on the user is transmittedfrom the recognition device to the external server.
 9. A display methodof a display device comprising a display unit including a plurality ofsub display modules connected to each other, the method comprising:obtaining data on a user located at a front of the display unit througha recognition device; analyzing the data on the user to generate a userinformation, the user information including at least one of an age ofthe user, a location of the user, a distance from the user and a gestureor voice of the user; adjusting a number of sub display modules whichare operated among the plurality of sub display modules to change a sizeof a display screen based on the user information, the sub displaymodules which are operated corresponding to the location of the user;and operating the adjusted sub display modules to display an image onthe adjusted sub display modules.
 10. The display method according toclaim 9, further comprising: changing a size of a text in the image;changing a contour between objects in the image; rotating the displayunit; or performing a conversion between a three-dimensional image and atwo-dimensional image.
 11. The display method according to claim 9,further comprising: converting a two-dimensional image processed by thecontrol unit into a three-dimensional image, wherein a depth of thethree-dimensional image is changed according to the user information.